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1  blank  line 


Formal  methods  are  mathematically  based  techniques,  often  supported  by  reasoning  tools  that  can 
-1  in.  — ►  offer  a  rigorous  and  effective  way  to  model,  design  and  analyze  computer  systems.  This  report  summarizes 
the  results  of  an  independent  study  of  12  cases  in  which  formal  methods  were  applied  to  the  construction 
of  industrial  systems.  A  major  conclusion  of  the  study  is  that  formal  methods,  while  still  immature  in 
certain  important  respects,  are  beginning  to  be  used  seriously  and  successfully  by  industry  to  design  and 
develop  computer  systems. 


Canada’s  Atomic  Energy  Control  Board  (AECB),  the  U.S.  National  Institute  of  Science  and  Technology 
(NIST),  and  the  U.S.  Naval  Research  Laboratory  (NRL)  commissioned  this  study  to  determine  the 
industrial  track  record  of  formal  methods  to  date  and  to  assess  the  efficacy  of  formal  methods  for  meeting 
their  needs.  The  study  had  three  main  objectives: 


1.  To  better  inform  deliberations  within  industry  and  government  on  standards  and  regulations; 

2.  To  provide  an  authoritative  record  on  the  practical  experience  of  formal  methods  to  date;  and 

3.  To  suggest  areas  where  further  research  and  technology  development  are  needed. 

These  objectives  have  been  accomplished  through  the  publication  of  this  report.  The  report  consists 
of  two  volumes  and  this  Executive  Summary.  The  first  volume  is  the  analysis  of  the  supporting  data 
contained  in  the  second  volume.  Volume  One  includes  a  discussion  on  formal  methods  and  a  brief 
characterization  of  the  formal  and  related  methods  used  in  the  cases,  a  summary  of  the  12  cases,  a 
description  of  the  methodology  used  in  the  survey,  a  cluster-by-cluster  analysis  of  the  data,  a  discussion 
on  the  key  events  and  timing  associated  with  each  case,  and  an  analysis  of  our  formal  methods  R&D 
summary;  it  concludes  with  the  findings,  observations,  and  conclusions.  The  appendixes  to  Volume  One 
contain  brief  biographies  of  the  authors,  brief  characterizations  of  11  formal  methods  used  in  the  cases, 
the  initial  questionnaire,  the  questionnaire  used  in  our  structured  interviews,  and  acknowledgements. 
Volume  Two  contains  detailed  supporting  data  on  the  12  cases. 


Through  these  means,  the  sponsors  have  been  provided  with  an  organized  body  of  systematic 
information,  assessments,  evaluations  and  observations  that  interpret  and  extrapolate  useful  data  on  cases 
of  significant  industrial  scale  and  applicability  to  real-world  products. 


This  Executive  Summary  presents  the  following: 


1 .  A  summary  of  the  major  findings  of  the  study. 

2.  Recommendations  for  continuing  R&D  in  formal  methods. 

FINDINGS  AND  RECOMMENDATIONS 


The  following  conclusions  are  the  result  of  applying  the  expertise  of  the  authors  in  analyzing  the 


cases. 

i 


0.75  in. 

I 


E-l 

1 0.5  in. 


1 


Executive  Summary 


10 


Calderwood  and  Staffieri 


FIRST  PAGE  OF  TEXT 


The  FIRST  PAGE  OF  TEXT  is  different  from  the  succeeding  text  pages.  The  page  number  for  only  the 
first  page  is  centered  0.5  in.  from  the  bottom  and  is  set  in  1 1-point  Times  Roman  using  as  Arabic  “1.”  (Page 
numbers  on  succeeding  pages  are  contained  in  the  headers.)  There  is  no  header  on  this  page. 


Margins — 1st  Page 


Top 

Bottom 

Left 

Right 

Fonts 

Title 

Text 

Heading  Level  1 
Heading  Level  2 
Heading  Level  3 
Heading  Level  4 
Heading  Level  5 
Heading  Level  6 


Inches 

2 

0.75 

1 

1 


TIMES  ROMAN  BOLD  12-POINT  FULL  CAPS 

Times  Roman  1 1  point 

TIMES  ROMAN  BOLD  11-POINT  FULL  CAPS 
Times  Roman  Bold  11-point  Initial  Caps 

Times  Roman  11 -point  Italic 

Times  Roman  Bold  11-point  Initial  Caps  Indented 

Times  Roman  1 1-point  Indented  Initial  Caps — ran  in  with  paragraph. 
Times  Roman  1 1 -point  Indented  Initial  Cap  of  1st  word — run  in  with 
paragraph. 


Headers 

There  is  no  header  on  the  first  page  of  text. 

Footer 

The  “Manuscript  approved  [date]”  footer  appears  at  the  bottom  of  the  first  page  of  text.  It  is  preceded  by 
a  0.007-in.  thick  horizontal  hairline.  This  line  is  0.75-in.  long  followed  by  a  hard  return.  The  text  is  9-point 
Times  Roman  flush  left  under  the  line  and  is  followed  by  two  hard  returns.  Turn  this  footer  off  after  page  1 
for  the  remainder  of  the  document. 

The  “Manuscript  approved  [date]”  is  taken  from  the  Publication  Approval  Form  and  is  the  date  the 
Division  Superintendent  signed  off  on  the  manuscript. 

Vertical  Spacing 

There  are  two  blank  lines  between  the  title  and  the  start  of  the  text.  There  is  one  blank  line  between 
paragraphs. 

There  is  one  blank  line  between  headings  levels  1, 2, 3,  and  4  and  the  text  following  these  headings.  The 
text  begins  on  the  same  line  after  heading  levels  5  and  6. 


NRL  Report  Formats 


11 


Text:  11  pt 


No  header  on 
this  page 


12  pt.  bold 
centered 


AN  INTERNATIONAL  SURVEY  OF  INDUSTRIAL  APPLICATIONS 
OF  FORMAL  METHODS  / 


VOLUME  1  X 

PURPOSE,  APPROACH,  ANALYSIS  AND  CONCLUSIONS 


|  2  blank  lines 
INTRODUCTION 


1  blank  line 


Formal  methods  are  mathematically  based  techniques,  often  supported  by  reasoning  tools  that  can 
N  offer  a  rigorous  and  effective  way  to  model,  design  and  analyze  computer  systems.  The  purpose  of  this  M 
study  is  to  evaluate  international  industrial  experience  in  using  formal  methods.  The  cases  selected  are, 
we  believe,  representative  of  industrial-grade  projects  and  span  a  variety  of  application  domains.  The 
study  had  three  main  objectives: 

•  To  better  inform  deliberations  within  industry  and  government  on  standards  and  regulations; 

•  To  provide  an  authoritative  record  on  the  practical  experience  of  formal  methods  to  date;  and 

•  To  suggest  areas  where  future  research  and  technology  development  are  needed. 

This  study  was  undertaken  by  three  experts  in  formal  methods  and  software  engineering:  Dan 
Craigen  of  ORA  Canada,  Susan  Gerhart  of  Applied  Formal  Methods,  and  Ted  Ralston  of  Ralston 
Research  Associates.  Robin  Bloomfield  of  Adelard  was  involved  with  the  Darlington  Nuclear  Generat¬ 
ing  Station  Shutdown  System  case.  Brief  biographies  of  the  authors  are  included  in  Appendix  A. 

Support  for  this  study  was  provided  by  organizations  in  Canada  and  the  United  States.  The  Atomic 
Energy  Control  Board  of  Canada  (AECB)  provided  support  for  Dan  Craigen  and  for  the  technical 
editing  provided  by  Karen  Summerskill.  The  Naval  Research  Laboratory  (NRL),  A^shington,  DC, 
provided  support  for  all  three  authors.  The  U.S.  National  Institute  of  Standards  and  Technology  (NIST) 
provided  support  for  Ted  Ralston. 

This  final  report  consists  of  two  volumes.  This  first  volume  describes  the  study,  formal  methods, 
the  cases  that  were  studied,  our  approach  to  performing  the  study,  and  our  analysis,  findings,  and 
conclusions. 

The  second  volume  of  the  final  report  provides  the  details  on  the  case  studies.  For  each  of  the  case 
studies,  we  present  a  case  description,  summarize  the  information  obtained  (from  interviews  and  the 
literature),  provide  an  evaluation  of  the  case,  highlight  R&D  issues  pertaining  to  formal  methods,  and 
provide  some  conclusions.  Earlier  drafts  of  the  case  studies  were  reviewed  by  the  relevant  participants. 

From  the  authors’  analysis  of  the  12  cases  and  the  stated  R&D  needs  from  those  we  interviewed, 
other  areas  are  suggested  for  future  R&D.  These  areas  are  drawn  from  the  particular  set  of  cases  that 
we  studied 
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FORMAL  METHODS 


Level  1 1 


Before  proceeding,  we  provide  an  historical  perspective,  explain  the  term  “formal  methods”  and  intro¬ 
duce  the  broad  spectrum  of  formal  methods  techniques  that  are  represented  in  the  survey. 


Historical  Perspective  |  Level  2 


For  over  two  decades,  researchers  have  explored  topics  in  the  mathematics  of  program  synthesis  and 
analysis.  The  article  “Assigning  Meaning  to  Programs”  [Floyd  1968]  stated  the  goal  of  both  (1)  semantics 
of  programming  languages,  and  (2)  specification  and  reasoning  about  individual  programs.  This  goal  evolved 
into  the  key  idea  of  inductive  assertions  then  defining  both  language  semantics  and  program  meaning  by 
relationships  among  preconditions,  program  statements,  and  postconditions.  The  intriguing  possibility  of 
mechanical  proof  of  programs,  or  alternatively,  heuristic  generation  of  programs,  yielded  many  exploratory 
systems  and  theoretical  insights.  Two  barriers  to  practical  application  arose:  (1)  it  was  difficult  to  capture 
the  full  semantic  content  of  programming  languages  and  operating  environments,  and  (2)  it  was  a  constant 
challenge  to  express  the  functional  and  nonfunctional  intent  for  a  program  in  its  context  of  use. 


Important  Concepts  \  Level  5 


Research  led  to  many  important  concepts:  formal  definitions  of  complex  language  features  and  identi¬ 
fication  of  pitfalls  of  unnecessary  and  overly  complex  features;  specification  languages  for  abstract  data 
types,  concurrent  processes,  and  abstract  machines;  a  theory  of  abstraction  behind  hierarchical  system 
structures;  mechanizable  logics  that  permitted  computational  reasoning  about  program  properties;  and 
theories  of  domains  such  as  security,  synchronous  clocks,  microprocessors,  and  compiling.  Practical  appli¬ 
cations  were  found  in  these  domains  and  small-to-medium  scale  examples  were  worked  out.  Industrializa¬ 
tion  began  in  the  U.S.  about  a  decade  ago  through  the  government  mandate  of  certification  of  security 
properties. 

Practice  went  a  different  route.  Verification  was  achieved  (and  defined)  through  case-based  reasoning 
(i.e.,  testing)  with  numerous  criteria  and  strategies  for  good  testing  practice  (primarily  functional  and 
structural  coverage).  Reviews  provided  the  primary  means  of  intellectual  control:  mental  checking  of  desir¬ 
able  properties  of  systems  under  development  and  the  concomitant  communication  among  stakeholders 
(such  as  managers,  designers,  testers,  and  documented).  Heuristic  methodologies  for  structured  require¬ 
ments  analysis  and  design  offered  additional  guidance  toward  systems  that  captured  the  conventional  wis¬ 
dom  of  good  structure  and  provided  a  common  means  of  communication. 

Verification  I  Level  4 1 


Researchers  developed  a  theoretical  base  for  testing  and  the  results,  although  mostly  negative,  suggested 
various  heuristics  for  testing  that  more  closely  approximated  an  ideal  where  each  test  case  meant  something 
with  some  chance  of  revealing  errors  or  demonstrating  new  evidence  of  correctness.  The  following  para¬ 
graphs  elaborate  on  this  information. 

Laval  5|  Heuristic  Methodologies— Heuristic  methodologies  from  practice  never  gained  much  research  attention 

although  abstract  data  types  gave  rise  to  object-oriented  languages  and  methods  to  add  even  more  structure 
and  support  to  heuristic  system  development.  Effort  in  this  area  is  somewhat  limited  and  should  be  expand¬ 
ed  for  additional  analysis.  A  number  of  important  concepts  and  their  interrelationships  need  to  be  explored. 
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Theoretical  results— Theoretical  results  have  also  played  a  role  in  system  development  (e.g.,  data 
compression,  error  correction,  and  encryption  algorithms  for  disk  and  network  storage  and  data  struc¬ 
tures  permit  representation  and  searching  of  data  bases  and  processing  of  visual  images). 


-1  in. — ^1 


Other  complexities— Especially  demanding  are  theories  and  strategies  for  managing  distributed  com¬ 
putation  and  data  on  both  physically  distributed  resources  and  multiprocessor  computing  systems. 

No  matter  what  technical  approach  is  applied  in  software  development,  common  information  pro¬ 
cessing  needs  arise:  maintaining  consistency  among,  and  intelligibility  of,  an  interwoven  mass  of  docu¬ 
ments  expressing  the  points  of  view  of  many  stakeholders,  with  constant  change  in  content  and  often 
change  in  structure  of  that  mass,  while  the  set  of  stakeholders  also  changes  over  what  may  be  many  years 
of  a  system’s  life.  Programming  environments  have  evolved  to  address  this  need:  structured  editors, 
configuration  management,  database  representations,  graphical  interfaces,  and  ways  of  coordinating  work 
flow  among,  as  well  as  work  products  of  groups  of  system  stakeholders.  Particularly  important  are  those 
assets  that  are  viewed  as  worthy  of  use  beyond  their  project  context  (e.g.,  software  components,  docu¬ 
ment  templates,  review  guidebooks,  error  and  productivity  data).  More  research  will  be  done  in  this  area 
in  the  future. 


Thread  in  Practice 

Yet  another  thread  in  practice  has  been  the  greater  attention  forced  onto  the  process  aspects  of  system 
development:  how  an  organization  manages  and  improves  its  infrastructure  and  specific  procedures. 
While  the  logic-based  form  of  mathematical  approaches  to  system  description  was  maturing,  so  was 
another  form:  statistical  reasoning  about  errors  and  growth  of  reliability  over  time,  with  the  objective  of 
introducing  industrial  quality  control  and  assurance  practices. 

Thus  we  have  the  setting  for  this  study  and  the  present  report:  mathematical  techniques  have  been 
maturing  for  25  years  while  non-mathematical  techniques  and  general  concerns  for  process  have  driven 
the  practice.  In  the  past  five  years,  sparse  anecdotal  evidence  indicated  that  formal  methods  were  begin¬ 
ning  to  be  used  in  industrial  practice,  leading  to  sponsorship  of  the  present  study  to  determine  systemat¬ 
ically  and  factually  where  these  applications  were  occurring,  why,  and  how  the  methods  were  being  used. 

What  Are  Formal  Methods? 

Definitions  of  formal  methods  abound.  In  the  FM89  report  (Craigen  and  Summerskill  1989),  formal 
methods  were  defined  as: 

“Methods  that  add  mathematical  rigor  to  the  development,  analysis,  and  operation  of  computer 

systems  and  to  applications  based  thereupon.” 


“...are  nothing  but  the  application  of  applied  mathematics— in  this  case,  formal  logic— to  the  design 
and  analysis  of  software-intensive  systems.” 


In  more  concrete  terms,  there  has  been  a  tendency,  on  the  part  of  the  formal  methods  community,  to 
define  formal  methods  in  terms  of  a  Hilbert-style  axiomatization.  For  example,  Robin  Bloomfield  has 
defined  formal  methods  as  follows: 


“A  software  specification  and  production  method,  based  on  a  mathematical  system,  that  comprises 
the  following: 
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PATTERN  RECOGNITION  ALGORITHM 


Suppose  we  have  a  digitized  I  xj  image  g  and  that  this  is  convolved  with  a  mask  or  kernel  k  of  size 
(2N  +  1)  x  (2N+ 1)  to  form  an  unsealed  image  h.  The  variables  involved  are  defined  by  Table  1.  The  process 
of  optimization,  as  shown  in  Fig.  1,  comprises  a  search  for  the  mask  &opt  in  a  domain,  or  set  of  acceptable 
masks,  K  for  which  f(G,  k)  is  maximum. 


3  blank  lines 


t 


0.5  in. 


Table  1  — Definitions  of  Variables 


T 


0.25  in. 


Object 

Format 

Class* 

Domainf 

Original  Image 

g  =  g(i.j) 

g. j  e  Z 

0  <g  <  G 
l<i<7 

1  <j  <7 

Convolution  Mask 

k  =  k(m,  n) 

ke  R 

-K<k<K 

m,nE  Z 

-N<m<N 

*Z  represents  the  set  of  all  integers  and  R  the  set  of  all  real  numbers 

tG  is  the  maximum  grey  level  in  g,  and  K  is  the  maximum  absolute  value  for  an  element  of  k. 


0.5  in. 


N  N 

h(i,j)=  X  X  8(i+m.j+n)- 

m=-N  n=-N 


(1) 


Where  a  mask  is  used  as  a  feature  detector  (as  in  the  current  project),  it  is  normal  to  apply  the  zero-sum 
constraint 


N  N 

X  X  k(m' n)  =  °’  (2) 

m=-N  n=-N 

to  prevent  response  to  a  uniformly  gray  image. 
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SOFTWARE  COST  REDUCTION  (SCR) 


1  blank  line 


The  description  presented  here  is  based  on  our  understanding  of  the  work  performed  at  Ontario  Hydro 
(on  the  Darlington  shutdown  system).  This  work  includes  evolutionary  enhancements  to  (Henry  1978)  by 
Pamas,  his  colleagues,  and  others. 

How  the  Method  Works 


Representations  Used 

Tabular  representations  of  software  requirements  (described  in  a  blackbox  manner)  and  of  program 
functions.  In  the  latter  case,  the  tables  are  called  “program  function  tables”  and,  essentially,  are  “strongest 
post  condition”  descriptions  of  a  procedure.  The  strongest  post  condition  explicitly  states  how  a  variable  is 
modified  by  a  procedure.  Also  included  are  “linkage  tables.” 

The  tabular  approach  to  describing  software  requirements  is  derived  from  the  work  performed  at  NRL. 
For  this  reason,  we  call  the  approach  “SCR-style.”  The  derivation  of  program  function  tables  and  the  use 
of  proofs  (described  below)  were  not  part  of  the  initial  work  on  the  SCR. 

Steps  Performed 

The  Darlington  project  was  generally  a  reverse  engineering  exercise.  The  code  already  existed  when 
they  developed  the  specifications  and  other  tables.  Proofs  were  required  to  demonstrate  that  a  routine 
specification  was  equivalent  to  the  program  function  description.  Proofs  were  performed  by  hand,  though 
it  could  be  mechanized. 

Code  linkage  tables  identify  how  the  outputs  of  a  program  function  table  may  be  used  as  inputs  to  other 
program  function  tables.  It  is  through  code  linkage  tables  that  all  the  physical  inputs  and  program  variables 
that  have  an  effect  on  a  physical  output  are  identified.  Similar  linkage  tables  exist  for  the  specification. 
Hence,  linkage  tables  help  to  modularize  the  descriptions  of  the  specification  and  code. 

Interaction  between  asynchronous  processes  was  viewed  as  interprocess  I/O  occurring  via  shared  data 
objects.  Each  process  had  its  own  specifications  and  program  function  tables.  Processes  were  analyzed 
separately  and  the  process  interaction  was  handled  separately  from  program  function  table  analysis. 

Tools  Applied 

No  specific  tools  exist  to  support  the  SCR-style  of  documentation.  Tools  are  expected  to  be  developed. 
The  strongest  post  condition  explicitly  states  how  a  variable  is  modified  by  a  procedure. 
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EXECUTIVE  SUMMARY  (U) 


(U)  INTRODUCTION 

(U)  Formal  methods  are  mathematically  based  techniques,  often  supported  by  reasoning  tools  that 
can  offer  a  rigorous  and  effective  way  to  model,  design  and  analyze  computer  systems.  This  report 
summarizes  the  results  of  an  independent  study  of  12  cases  in  which  formal  methods  were  applied  to  the 
construction  of  industrial  systems.  A  major  conclusion  of  the  study  is  that  formal  methods,  while  still 
immature  in  certain  important  respects,  are  beginning  to  be  used  seriously  and  successfully  by  industry  to 
design  and  develop  computer  systems. 

(U)  Canada’s  Atomic  Energy  Control  Board  (AECB),  the  U.S.  National  Institute  of  Science  and 
Technology  (NIST),  and  the  U.S.  Naval  Research  Laboratory  (NRL)  commissioned  this  study  to  determine 
the  industrial  track  record  of  formal  methods  to  date  and  to  assess  the  efficacy  of  formal  methods  for 
meeting  their  needs.  The  study  had  three  main  objectives: 

1.  (U)  To  better  inform  deliberations  within  industry  and  government  on  standards  and  regulations; 

2.  (U)  To  provide  an  authoritative  record  on  the  practical  experience  of  formal  methods  to  date;  and 

3.  (U)  To  suggest  areas  where  further  research  and  technology  development  are  needed. 

(U)  These  objectives  have  been  accomplished  through  the  publication  of  this  report.  The  report 
consists  of  two  volumes  and  this  Executive  Summary.  The  first  volume  is  the  analysis  of  the  supporting 
data  contained  in  the  second  volume.  Volume  One  includes  a  discussion  on  formal  methods  and  a  brief 
characterization  of  the  formal  and  related  methods  used  in  the  cases,  a  summary  of  the  12  cases,  a 
description  of  the  methodology  used  in  the  survey,  a  cluster-by-cluster  analysis  of  the  data,  a  discussion  on 
the  key  events  and  timing  associated  with  each  case,  and  an  analysis  of  our  formal  methods  R&D 
summary;  it  concludes  with  the  findings,  observations,  and  conclusions.  The  appendixes  to  Volume  One 
contain  brief  biographies  of  the  authors,  brief  characterizations  of  11  formal  methods  used  in  the  cases,  the 
initial  questionnaire,  the  questionnaire  used  in  our  structured  interviews,  and  acknowledgements.  Volume 
Two  contains  detailed  supporting  data  on  the  12  cases. 

(U)  Through  these  means,  the  sponsors  have  been  provided  with  an  organized  body  of  systematic 
information,  assessments,  evaluations  and  observations  that  interpret  and  extrapolate  useful  data  on  cases 
of  significant  industrial  scale  and  applicability  to  real-world  products. 

(U)  This  Executive  Summary  presents  the  following: 

1.  (U)  A  summary  of  the  major  findings  of  the  study. 

2.  (U)  Recommendations  for  continuing  R&D  in  formal  methods. 

(U)  FINDINGS  AND  RECOMMENDATIONS 

(U)  The  following  conclusions  are  the  result  of  applying  the  expertise  of  the  authors  in  analyzing  the 
cases. 

E-l 
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AN  INTERNATIONAL  SURVEY  OF  INDUSTRIAL  APPLICATIONS 
OF  FORMAL  METHODS  (U) 

VOLUME  1 

PURPOSE,  APPROACH,  ANALYSIS  AND  CONCLUSIONS  (U) 


(U)  INTRODUCTION 

(U)  Formal  methods  are  mathematically  based  techniques,  often  supported  by  reasoning  tools  that 

can  offer  a  rigorous  and  effective  way  to  model,  design  and  analyze  computer  systems.  The  purpose  of  this 
study  is  to  evaluate  international  industrial  experience  in  using  formal  methods.  The  cases  selected  are,  we 
believe,  representative  of  industrial-grade  projects  and  span  a  variety  of  application  domains.  The  study 
had  three  main  objectives: 

•  (U)  To  better  inform  deliberations  within  industry  and  government  on  standards  and  regulations; 

•  (U)  To  provide  an  authoritative  record  on  the  practical  experience  of  formal  methods  to  date;  and 

•  (U)  To  suggest  areas  where  future  research  and  technology  development  are  needed. 

(U)  This  study  was  undertaken  by  three  experts  in  formal  methods  and  software  engineering:  Dan 
Craigen  of  ORA  Canada,  Susan  Gerhart  of  Applied  Formal  Methods,  and  Ted  Ralston  of  Ralston  Research 
Associates.  Robin  Bloomfield  of  Adelard  was  involved  with  the  Darlington  Nuclear  Generating  Station 
Shutdown  System  case.  Brief  biographies  of  the  authors  are  included  in  Appendix  A. 

(U)  Support  for  this  study  was  provided  by  organizations  in  Canada  and  the  United  States.  The 
Atomic  Energy  Control  Board  of  Canada  (AECB)  provided  support  for  Dan  Craigen  and  for  the  technical 
editing  provided  by  Karen  Summerskill.  The  Naval  Research  Laboratory  (NRL),  Washington,  DC,  provid¬ 
ed  support  for  all  three  authors.  The  U.S.  National  Institute  of  Standards  and  Technology  (NIST)  provided 
support  for  Ted  Ralston. 

(U)  This  final  report  consists  of  two  volumes.  This  first  volume  describes  the  study,  formal  methods, 
the  cases  that  were  studied,  our  approach  to  performing  the  study,  and  our  analysis,  findings,  and  conclu¬ 
sions. 

(U)  The  second  volume  of  the  final  report  provides  the  details  on  the  case  studies.  For  each  of  the 
case  studies,  we  present  a  case  description,  summarize  the  information  obtained  (from  interviews  and  the 
literature),  provide  an  evaluation  of  the  case,  highlight  R&D  issues  pertaining  to  formal  methods,  and 
provide  some  conclusions.  Earlier  drafts  of  the  case  studies  were  reviewed  by  the  relevant  participants. 

(U)  From  the  authors’  analysis  of  the  12  cases  and  the  stated  R&D  needs  from  those  we  interviewed, 
other  areas  are  suggested  for  future  R&D.  These  areas  are  drawn  from  the  particular  set  of  cases  that  we 
studied. 
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(U)  FORMAL  METHODS 

(U)  Before  proceeding,  we  provide  an  historical  perspective,  explain  the  term  “formal 
methods”  and  introduce  the  broad  spectrum  of  formal  methods  techniques  that  are  represent¬ 
ed  in  the  survey. 

(U)  Historical  Perspective 

(U)  For  over  two  decades,  researchers  have  explored  topics  in  the  mathematics  of  program 
synthesis  and  analysis.  The  article  “Assigning  Meaning  to  Programs”  [Floyd  1968]  stated  the 
goal  of  both  (1)  semantics  of  programming  languages,  and  (2)  specification  and  reasoning 
about  individual  programs.  This  goal  evolved  into  the  key  idea  of  inductive  assertions  then 
defining  both  language  semantics  and  program  meaning  by  relationships  among  precondi¬ 
tions,  program  statements,  and  postconditions.  The  intriguing  possibility  of  mechanical  proof 
of  programs,  or  alternatively,  heuristic  generation  of  programs,  yielded  many  exploratory 
systems  and  theoretical  insights.  Two  barriers  to  practical  application  arose:  (1)  it  was  diffi¬ 
cult  to  capture  the  full  semantic  content  of  programming  languages  and  operating  environ¬ 
ments,  and  (2)  it  was  a  constant  challenge  to  express  the  functional  and  nonfunctional  intent 
for  a  program  in  its  context  of  use. 

(U)  Important  Concepts 

(U)  Research  led  to  many  important  concepts:  formal  definitions  of  complex  language 
features  and  identification  of  pitfalls  of  unnecessary  and  overly  complex  features;  specifica¬ 
tion  languages  for  abstract  data  types,  concurrent  processes,  and  abstract  machines;  a  theory 
of  abstraction  behind  hierarchical  system  structures;  mechanizable  logics  that  permitted  com¬ 
putational  reasoning  about  program  properties;  and  theories  of  domains  such  as  security, 
synchronous  clocks,  microprocessors,  and  compiling.  Practical  applications  were  found  in 
these  domains  and  small-to-medium  scale  examples  were  worked  out.  Industrialization  began 
in  the  U.S.  about  a  decade  ago  through  the  government  mandate  of  certification  of  security 
properties. 

(U)  Practice  went  a  different  route.  Verification  was  achieved  (and  defined)  through  case- 
based  reasoning  (i.e.,  testing)  with  numerous  criteria  and  strategies  for  good  testing  practice 
(primarily  functional  and  structural  coverage).  Reviews  provided  the  primary  means  of  intel¬ 
lectual  control:  mental  checking  of  desirable  properties  of  systems  under  development  and  the 
concomitant  communication  among  stakeholders  (such  as  managers,  designers,  testers,  and 
documenters).  Heuristic  methodologies  for  structured  requirements  analysis  and  design  of¬ 
fered  additional  guidance  toward  systems  that  captured  the  conventional  wisdom  of  good 
structure  and  provided  a  common  means  of  communication. 

(U)  Verification 

(U)  Researchers  developed  a  theoretical  base  for  testing  and  the  results,  although  mostly 
negative,  suggested  various  heuristics  for  testing  that  more  closely  approximated  an  ideal 
where  each  test  case  meant  something  with  some  chance  of  revealing  errors  or  demonstrating 
new  evidence  of  correctness.  The  following  paragraphs  elaborate  on  this  information. 

(U)  Heuristic  Methodologies— Heuristic  methodologies  from  practice  never  gained  much 
research  attention  although  abstract  data  types  gave  rise  to  object-oriented  languages  and 
methods  to  add  even  more  structure  and  support  to  heuristic  system  development.  Effort  in 
this  area  is  somewhat  limited  and  should  be  expanded  for  additional  analysis.  A  number  of 
important  concepts  and  their  interrelationships  need  to  be  explored. 
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(U)  Theoretical  results— Theoretical  results  have  also  played  a  role  in  system  develop¬ 
ment  (e.g.,  data  compression,  error  correction,  and  encryption  algorithms  for  disk  and 
network  storage  and  data  structures  permit  representation  and  searching  of  data  bases  and 
processing  of  visual  images). 

(U)  Other  complexities— Especially  demanding  are  theories  and  strategies  for  managing 
distributed  computation  and  data  on  both  physically  distributed  resources  and  multiprocessor 
computing  systems. 

(U)  No  matter  what  technical  approach  is  applied  in  software  development,  common 
information  processing  needs  arise:  maintaining  consistency  among,  and  intelligibility  of,  an 
interwoven  mass  of  documents  expressing  the  points  of  view  of  many  stakeholders,  with 
constant  change  in  content  and  often  change  in  structure  of  that  mass,  while  the  set  of 
stakeholders  also  changes  over  what  may  be  many  years  of  a  system’s  life.  Programming 
environments  have  evolved  to  address  this  need:  structured  editors,  configuration  manage¬ 
ment,  database  representations,  graphical  interfaces,  and  ways  of  coordinating  work  flow 
among,  as  well  as  work  products  of  groups  of  system  stakeholders.  Particularly  important  are 
those  assets  that  are  viewed  as  worthy  of  use  beyond  their  project  context  (e.g.,  software 
components,  document  templates,  review  guidebooks,  error  and  productivity  data).  More 
research  will  be  done  in  this  area  in  the  future. 

(U)  Thread  in  Practice 

(U)  Yet  another  thread  in  practice  has  been  the  greater  attention  forced  onto  the  process 
aspects  of  system  development:  how  an  organization  manages  and  improves  its  infrastructure 
and  specific  procedures.  While  the  logic-based  form  of  mathematical  approaches  to  system 
description  was  maturing,  so  was  another  form:  statistical  reasoning  about  errors  and  growth 
of  reliability  over  time,  with  the  objective  of  introducing  industrial  quality  control  and  assur¬ 
ance  practices. 

(U)  Thus  we  have  the  setting  for  this  study  and  the  present  report:  mathematical  tech¬ 
niques  have  been  maturing  for  25  years  while  non-mathematical  techniques  and  general 
concerns  for  process  have  driven  the  practice.  In  the  past  five  years,  sparse  anecdotal  evi¬ 
dence  indicated  that  formal  methods  were  beginning  to  be  used  in  industrial  practice,  leading 
to  sponsorship  of  the  present  study  to  determine  systematically  and  factually  where  these 
applications  were  occurring,  why,  and  how  the  methods  were  being  used. 

(U)  What  Are  Formal  Methods? 

(U)  Definitions  of  formal  methods  abound.  In  the  FM89  report  (Craigen  and  Summerskill 
1989),  formal  methods  were  defined  as: 

“Methods  that  add  mathematical  rigor  to  the  development,  analysis,  and  operation 
of  computer  systems  and  to  applications  based  thereupon.  ” 

“...are  nothing  but  the  application  of  applied  mathematics-in  this  case,  formal 
logic-to  the  design  and  analysis  of  software-intensive  systems.” 

(U)  In  more  concrete  terms,  there  has  been  a  tendency,  on  the  part  of  the  formal 
methods  community,  to  define  formal  methods  in  terms  of  a  Hilbert-style  axiomatization. 
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(U)  PATTERN  RECOGNITION  ALGORITHM 


(U)  Suppose  we  have  a  digitized  I  xJ  image  g  and  that  this  is  convolved  with  a  mask  or  kernel  k  of  size 
( 2N  +  1)  x  (2N  + 1 )  to  form  an  unsealed  image  h.  The  variables  involved  are  defined  by  Table  1 .  The  process 
of  optimization,  as  shown  in  Fig.  1,  comprises  a  search  for  the  mask  k0 pt  in  a  domain,  or  set  of  acceptable 
masks,  K  for  which  f(G,  k)  is  maximum. 


Table  1  —  (U)  Definitions  of  Variables 


Object 

Format 

Class* 

Domainf 

Original  Image 

g  =  gtt,j) 

g,ue  z 

0<g<G 

1  <i  </ 
l^j^^ 

Convolution  Mask 

k  =  k(m,  n) 

kG  R 

-K<k<K 

m,  n  £  Z 

-N  <m  <N 

*Z  represents  the  set  of  all  integers  and  R  the  set  of  all  real  numbers 
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Fig.  1  -  (U)  Schematic  diagram  of  autonomous  target-detection  system 


(U)  The  convolution  operation  h  =  g  +  k  is  commonly  defined  by 
N  N 

h(i,j)=  X  X  S(i+ni,j+n).  (1) 

m--N  n=-N 

Where  a  mask  is  used  as  a  feature  detector  (as  in  the  current  project),  it  is  normal  to  apply  the  zero-sum 
constraint 

N  N 

X  X  k(m- n) = °-  (2) 

m=-N  n=~N 
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Appendix  A 

FORMAL  METHODS  TECHNIQUES  (U) 


(U)  SOFTWARE  COST  REDUCTION  (SCR) 

(U)  The  description  presented  here  is  based  on  our  understanding  of  the  work  performed  at 
Ontario  Hydro  (on  the  Darlington  shutdown  system).  This  work  includes  evolutionary 
enhancements  to  (Henry  1978)  by  Parnas,  his  colleagues,  and  others. 

(U)  How  the  Method  Works 

(U)  Representations  Used 

(U)  Tabular  representations  of  software  requirements  (described  in  a  blackbox  manner)  and 
of  program  functions.  In  the  latter  case,  the  tables  are  called  “program  function  tables”  and, 
essentially,  are  “strongest  post  condition”  descriptions  of  a  procedure.  The  strongest  post  condition 
explicitly  states  how  a  variable  is  modified  by  a  procedure.  Also  included  are  “linkage  tables.” 

(U)  The  tabular  approach  to  describing  software  requirements  is  derived  from  the  work 
performed  at  NRL.  For  this  reason,  we  call  the  approach  “SCR-style.”  The  derivation  of 
program  function  tables  and  the  use  of  proofs  (described  below)  were  not  part  of  the  initial  work 
on  die  SCR. 

(U)  Steps  Performed 

(U)  The  Darlington  project  was  generally  a  reverse  engineering  exercise.  The  code  already 
existed  when  they  developed  the  specifications  and  other  tables.  Proofs  were  required  to 
demonstrate  that  a  routine  specification  was  equivalent  to  the  program  function  description. 
Proofs  were  performed  by  hand,  though  it  could  be  mechanized. 

(U)  Code  linkage  tables  identify  how  the  outputs  of  a  program  function  table  may  be  used  as 
inputs  to  other  program  function  tables.  It  is  through  code  linkage  tables  that  all  the  physical 
inputs  and  program  variables  that  have  an  effect  on  a  physical  output  are  identified.  Similar 
linkage  tables  exist  for  the  specification.  Hence,  linkage  tables  help  to  modularize  the  descriptions 
of  the  specification  and  code. 

(U)  Interaction  between  asynchronous  processes  was  viewed  as  interprocess  I/O  occurring 
via  shared  data  objects.  Each  process  had  its  own  specifications  and  program  function  tables. 
Processes  were  analyzed  separately  and  the  process  interaction  was  handled  separately  from 
program  function  table  analysis. 

(U)  Tools  Applied 

(U)  No  specific  tools  exist  to  support  the  SCR-style  of  documentation.  Tools  are  expected  to 
be  developed. 
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Appendix  A 

FORMAL  METHODS  TECHNIQUES 

Unclassified  appendixes  to  classified  documents  may  be  printed  as  separate  documents  if 
desired;  page  and  item  classification  markings  are  omitted  in  this  case.  However,  almost  all 
appendixes  are  bound  in  the  document  and,  having  been  bound  together,  remain  together.  Thus, 
an  unclassified  appendix  in  a  classified  document  must  bear  page  security  markings  reflecting 
the  highest  security  classification  used  in  the  document.  If  the  appendixes  contain  no  classified 
material  (such  as  headings,  text,  tables,  and  figures),  they  must  begin  on  a  right-hand  page. 
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Appendix 

ADDITIONAL  TYPOGRAPHICAL  COMMENTS 


PAPER  SIZE  8.5x11  in. 


FONTS 


Use  a  Roman  typeface  such  as  Times  Roman,  New  Times  Roman,  or  CG  Times.  The  use  of  bold  and 
italic  fonts  of  these  typefaces  is  required  to  match  the  specified  format. 

Title  TIMES  ROMAN  BOLD  12-POINT  FULL  CAPS 

Text  Times  Roman  1 1  point 


Heading  Level  1 
Heading  Level  2 
Heading  Level  3 
Heading  Level  4 
Heading  Level  5 

Heading  Level  6 


TIMES  Roman  BOLD  11-POINT  FULL  CAPS 
Times  Roman  Bold  11-point  Initial  Caps 

Times  Roman  11 -point  Italic 

Times  Roman  Bold  11-point  Initial  Caps  Indented 

Times  Roman  1 1 -point  Indented  Initial  Caps — run  in  with 
paragraph. 

Times  Roman  1 1 -point  Indented  Initial  Cap  of  1st  word — run  in 
with  paragraph. 


Header  A — Author’s  name(s) 
Header  B — Brief  title 
Figure  captions 
Table  titles 

Footer  A — Manuscript  approved 


Times  Roman  10-point  italic 
Times  Roman  10-point  italic 
Times  Roman  9  point 
Times  Roman  1 1  point 
Times  Roman  9  point 


TABS  AND  INDENTS 


Although  tabs  and  indents  may  be  adjusted  to  fit  the  requirements  of  the  document,  the  following  are 
recommended  basic  spacings  for  tabs  and  indents: 


Inches 

1st 

0.273 

2nd 

0.438 

3rd 

0.921 

4th 

1.175 

5th 

2.0 

6th 

2.75 

JUSTIFICATION  Full 


SPACES 


At  end  of  sentence 

After  a  sequential  number 
(e.g.,  1.,  2.,  3.,  etc.) 


Single  space 
Em-space 


An  em-space  is  equal  to  the  type  size  in 
points.  An  em-space  for  1 1 -point  type  is 
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1 1  points  wide,  with  no  space  before  or  after.  (A  double  space  is 
acceptable.) 

DASHES 

Hyphenation 

After  a  figure  number 
in  a  caption 


To  show  a  range 
Double  dashes  (-) 


PAGE  NUMBERING 

Front  matter  Lowercase  Roman  numerals  centered  at  bottom  of  page  0.5 

in.  from  bottom  edge  set  in  11 -point  Times  Roman. 

Page  1  of  text  Arabic  “1”  centered  at  bottom  of  page,  0.5  in.  from  bottom 

edge  set  in  11-point  Times  Roman. 

All  other  text  pages  Arabic  numbers  part  of  headers  A  and  B  set  in  10  point 

Times  Roman  at  outer  margins:  odd  pages  (header  A)  are  flush 
right,  even  pages  (header  B)  are  flush  left. 

EXECUTIVE  SUMMARY  Arabic  numbers  starting  with  E-l  centered  at  bottom  of  page 

0.5  in.  from  bottom  edge  set  in  11-point  Times  Roman. 


MULTILINE  FIGURE  CAPTIONS 

If  a  figure  caption  is  two  lines  long,  the  second  line  is  centered  under  the  first  caption  line,  including  the 
figure  number.  If  the  figure  caption  is  three  or  more  lines  long,  the  lines  are  typed  full-figure  width,  same  as 
the  first  line  of  the  caption,  including  the  figure  number. 

MULTILINE  TABLE  TITLES 

If  a  table  title  is  more  than  one  line  long,  subsequent  lines  are  centered  under  the  complete  first  line  of 
the  title,  including  the  table  number. 


PARAGRAPH  NUMBERING 

Paragraphs  generally  are  not  numbered.  However,  if  paragraph  numbering  is  required  for  reference 
purposes,  then  use  the  numeric  decimal  system  as  follows  with  an  em-space  between  the  number  and  the 
heading  or  text. 


Use  a  regular  hyphen  or  dash 

Em-dash  An  em-dash  is  equal  to  the  type  size  in  points.  An 
em-dash  for  1 1 -point  type  is  1 1  points  wide.  Put  a 
space  before  and  after  the  em-dash. 

En-dash  An  en-dash  is  half  of  an  em  dash. 

Use  an  em-dash  ( — ). 


